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A system and methods through which pictorial views of an 
object’s configuration, arranged in a hierarchical fashion, are 
navigated by a person to establish a visual context within the 
configuration. The visual context is automatically translated 
by the system into a set of search parameters driving retrieval 
of structured data and content (images, documents, multime- 
dia, etc.) associated with the specific context. The system 
places “hot spots”, or actionable regions, on various portions 
of the pictorials representing the object. When a user interacts 
with an actionable region, a more detailed pictorial from the 
hierarchy is presented representing that portion of the object, 
along with real-time feedback in the form of a popup pane 
containing information about that region, and counts-by-type 
reflecting the number of items that are available within the 
system associated with the specific context and search filters 
established at that point in time. 
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IMAGE AND INFORMATION MANAGEMENT 
SYSTEM 

This application is a continuation of and claims the benefit 
of U.S. patent application Ser. No. 1 1/466,508, filed Aug. 23, 
2006 now U.S. Pat. No. 7,295,719, which is related to and 
claims the benefit of Provisional U.S. Patent Application Ser. 
No. 60/7 11,414 entitled “Automated Resource Planning Tool 
and User Interface”, filed Aug. 26, 2005, and Provisional U.S. 
Patent Application Ser. No. 60/711,413 entitled “Image and 
Information Management System”, filed Aug. 26, 2005, 
which are hereby incorporated by reference in their entirety. 

The invention described herein was made in the perfor- 
mance of work under NASA Contract No. NAS9-20000 and 
is subject to the provisions of Section 305 of the National 
Aeronautics and Space Act of 1958 (42 U.S.C. 2457). 

This application includes material which is subject to 
copyright protection. The copyright owner has no objection to 
the facsimile reproduction by anyone of the patent disclosure, 
as it appears in the Patent and Trademark Office files or 
records, but otherwise reserves all copyright rights whatso- 
ever. 

FIELD 

The present disclosure relates to the field of information 
systems, and more particularly provides a system and method 
through which images of and information about an object can 
be stored, organized, and searched to facilitate ease of 
retrieval and use. 

BACKGROUND 

The efficient storage, search, and retrieval of information 
has long been a problem. From filing cabinets full of paper to 
hard-drives full of data, individuals, businesses, and govern- 
ments have struggled to find good organization techniques. 
The advent of databases and enhanced search and indexing 
algorithms has made it easier to find desired electronic data. 
For example, Liu et al., U.S. Patent Application Publication 
No. 2005/0114325, published May 26, 2005, and Liu, U.S. 
Patent Application Publication No. 2005/0010553, published 
Jan. 13, 2005, disclose a semi-automatic means for annotat- 
ing multimedia objects which also permits retrieval of the 
multimedia objects. The annotation process annotates multi- 
media objects, such as digital images, with semantically rel- 
evant keywords. Such annotation is performed in the back- 
ground, with the results presented to the user for verification. 
The user can also add his or her own keywords or other 
annotation information. Similarly, Remsen et al., U.S. Patent 
Application Publication No. 2003/0167283, published Sep. 
4, 2003, discloses a system for managing taxonomic infor- 
mation regarding biological organisms that allows research- 
ers to quickly retrieve information on a given organism 
regardless of the name associated with the information. 

While such systems can be advantageous where the infor- 
mation to be cataloged is mostly text or includes textual 
metadata, searching a set of multimedia objects, such as 
images, sounds, or video, for a particular item of interest can 
be difficult. Stubler et al., U.S. Pat. No. 6,804,684, issued Oct. 
12, 2004, discloses a method for associating semantic infor- 
mation with multiple images in an image database environ- 
ment. Stubler discloses generating captions or semantic 
labels for an acquired imaged based upon similarities 
between the acquired image and one or more stored images, 
where the stored images have preexisting captions or labels. 
Similarly, DeBonet, U.S. Pat. No. 5,819,288, issued Oct. 6, 
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1998, discloses a statistically based image group descriptor 
for use in an image classification and retrieval system. DeBo- 
net discloses an apparatus and method for generating a 
semantically based, linguistically searchable, numeric 
5 descriptor of a pre-defined group of input images which can 
automatically classify individual images to allow for rapid 
identification of similar images. 

The systems described above facilitate searching by anno- 
tating a database entry with a keyword. Where the data may be 
to organized by a plurality of key words, or where the data may 
not have appropriate key words, database searches may not 
return the entire set of available information, and thus valu- 
able information may be effectively inaccessible when 
needed. Matsuzaki et al., U.S. Pat. No. 6,804,683, issued Oct. 
15 12, 2004, attempts to overcome those deficiencies by disclos- 
ing a means for retrieving similar images that allows a user to 
specify a “region of interest”, and compares a “feature quan- 
tity calculation” of the region to the quantity calculation of 
other images in the database. The search results are presented 
20 in order of decreasing similarity from the image database. 

It can clearly be difficult to find appropriate data in a 
database, and the techniques described above have been 
developed to improve search results. Once the search results 
are obtained, the next concern is the presentation of these 
25 search results. Baker, U.S. Patent Application Publication 
No. 2002/0047856, published Apr. 25, 2002, discloses a 
method of providing access to a collection of images through 
a single web page, which Baker refers to as “stacked images.” 
Baker discloses building a database that represents a plurality 
30 of separate images, and then displaying at least one of these 
images. When a user clicks the image, the next image in the 
“stack” is displayed. Lennon et al., U.S. Patent Application 
Publication No. 2003/0018607, published Jan. 23, 2003, dis- 
closes a system for allowing a user to search different meta- 
35 data collections over the Internet. Lennon discloses dynamic 
generation of an XML description of an associated metadata 
collection that reflects a structure by which the associated 
metadata collection may be browsed. This structure includes 
XML schema definitions of types for Categories, Subcatego- 
40 lies, Classes, and Images. Lennon discloses that each 
declared descriptor within the defined type inherits a visuall- 
dentifier attribute which is used by a media browser to provide 
visual representation of the content of the item. For example, 
if the item is an image then the visual Identifier attribute value 
45 will typically contain a URI of a thumbnail of the image; if the 
item is a Category, Subcategory, or Class, then the URI may 
point to an icon. If the attribute is not present, the media 
browser generates a visual identifier for the item from a 
provided textldentifier attribute value or from the name of the 
50 element. 

While several different techniques exist for the search, 
retrieval, and presentation of information, most are too cum- 
bersome to be useful in instances where time is of the essence, 
or where it is critical that the all relevant information be 
55 presented without overw helming the user. For example, Rem- 
sen’s system is intentionally over inclusive, and provides 
information about related species in addition to the species 
for which the user is searching. Similarly, Stubler’ s and 
DeBonet’ s search techniques are likely to both include infor- 
60 mation that is not relevant, and to overlook relevant images. 

SUMMARY 

Accordingly, the instant disclosure is directed to an image 
65 and information management system that substantially obvi- 
ates one or more of the problems due to limitations and 
disadvantages of the related art. As used herein, the term 
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image includes photographs, line art, computer renderings, 
animation, motion video, and the like, and is not intended as 
limited to a particular type of image. 

The present disclosure describes a system and methods for 
allowing users to access relevant information about an object 
quickly and easily. An object can be a physical item such as a 
spacecraft or aircraft, a piece of hardware or equipment, a 
facility, a building, or a piece of machinery, among others, or 
an abstract item such as a schematic, a flow diagram, an 
illustration, or a representation of a business function, and the 
like. 

The instant disclosure also describes a system and method 
for providing a graphical, drill-down interface through which 
information about an object can be accessed. 

The instant disclosure also describes a system and method 
for providing a structured interface through which informa- 
tion about an object can be captured and cataloged. 

Additional features and advantages will be set forth in the 
description which follows, and in part will be apparent from 
the description, or may be learned by practice of the disclo- 
sure. The objectives and other advantages of the disclosed 
image and information management system will be realized 
and attained by the structure particularly pointed out in the 
written description and claims hereof as well as the appended 
drawings. 

In many instances, the information storage and retrieval 
systems in the prior art are sufficient for business purposes. 
However, where time is of the essence, such as occurs in an 
assembly line; in the manufacture and maintenance of mis- 
sion-critical equipment such as aircraft or watercraft; during 
fire and other emergencies; and the like, it is critical that users 
be able to quickly and easily access all relevant information. 
By way of example, without limitation, any delays in the 
launch of a spacecraft can have significant economic impact 
upon the entire operation. From manpower costs due to 
missed launch windows and contract -related fines to addi- 
tional fuel and real estate costs, every minute launch is 
delayed has a significant economic impact. Similarly, where 
the object is not available for analysis, such as in an orbiting 
spacecraft, or where the object is effectively inaccessible, 
such as the wiring of an aircraft while the aircraft is in flight, 
the antenna array on a submarine while the submarine is at 
depth, or an underwater valve assembly on an oil rig, it can be 
advantageous to have well documented and readily accessible 
information. By way of example, without limitation, trouble- 
shooting on-orbit or in-flight anomalies in a spacecraft 
requires timely access to spacecraft configuration data in the 
form of imagery and information. Review of configuration 
imagery is critical to quickly developing in-flight repair pro- 
cedures. In these and other scenarios, it is essential that, when 
there is a problem, those addressing the problem have reliable 
access to the pertinent information about the object that is the 
subject of the problem. The instant disclosure provides a 
system and methods through which users can gain rapid and 
reliable access to such information. 

A graphical, image-based interface consisting of a hierar- 
chically structured collection of integrated pictorial views of 
an object’s configuration can be presented to one or more 
users. The hierarchy of pictorial views enables users to estab- 
lish a visual context within the object’s configuration that 
defines search parameters driving retrieval of information 
associated with the specific context. For example, users can 
access and refer to photographs, engineering drawings, line- 
art, or images of a spacecraft associated with a visual context 
established for the information retrieval search. The system 
can allow users to drill down quickly to a visual context 
within the object, such as drilling down to a faulty component 
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within the object, by simply clicking on or otherwise inter- 
acting with (henceforth referred to herein as clicking, for 
clarity) “hotspots”, actionable regions, or zones in an image 
or successive images. 

5 The interface can automatically translate the user’s inter- 
actions into search parameters, which are then used as a basis 
for information retrieval, including accessing a next-lower 
level of an image or “drilldown” hierarchy where appropriate. 
The retrieved information can include images of a selected 
to component, including photographs taken during the manu- 
facture and/or assembly of the individual component as well 
as those taken during the component’s installation. In addi- 
tion, information about the photographed component, such 
as, without limitation, part and serial number(s) of items 
15 within the component, schematics, dates of manufacture, 
installed software and version numbers, lot numbers, chemi- 
cal composition, test results, related tools, and the like can 
also be made available to the user. 

A standardized interface is also provided through which 
20 information about the component, including component pho- 
tographs, can be cataloged and stored. This embodiment can 
also integrate with and monitor other information databases, 
such as document management systems and, using exten- 
sible Markup Language (“XML”)-based tags, Standardized 
25 Generalized Markup Language (SGML) -based tags, key 
words, and the like stored within a document or in document- 
related metadata, automatically associate the document with 
appropriate information stored in the system. 

The image and information management system has appli- 
30 cation in numerous commercial and government settings. By 
way of example, without limitation, any commercial entity 
with requirements for visual, as-built, or operational configu- 
ration baselines can benefit from such a system. Such com- 
mercial entities may include, but are not limited to, compa- 
35 nies engaged in oil and gas extraction, mining, petroleum 
refining, chemical production, power generation or heavy 
construction who operate complex sets of equipment and 
facilities supporting their core business functions. These 
companies can benefit from the Visual Configuration Base- 
40 lines (“VCBs”) created and maintained through the system by 
making imagery, documentation, test results, test data, engi- 
neering models, additional items in inventory, and other criti- 
cal data readily available for troubleshooting, problem reso- 
lution, product and process design, operations or analytical 
45 purposes. 

The system also provides browser-integrated, in-line view- 
ing tools to support examination of high-resolution photo- 
graphs, and browser-integrated Digital Content Capture tools 
for assembling and storing information (structured data and 
50 multi-media content) associated with the object. The infor- 
mation is stored in relational form in a commercial, off the 
shelf (COTS) document management system. 

The system provides a very simple and reliable search/ 
retrieval interface by moving the burden of specifying the 
55 exact interrelated attribute search criteria from the user to the 
underlying search system with the user being responsible for 
inputting only a small number of commonly known search 
values in an easy and intuitive fashion. Once the user provides 
a minimum input set, the system transparently generates a 
60 fully qualified set of search criteria that accurately returns a 
result set corresponding to the visual context and filters estab- 
lished by the user. 

The image and information management system can func- 
tion as part of an overall tool set through which a space flight 
65 can be monitored and controlled from early conception and 
design phases through execution. Such a system preferably 
includes a tool for automating crew scheduling, wherein the 
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tool can automatically adjust crew schedules based on 
changes in circumstances, such as changes in vehicle attitude 
and trajectory. Additionally, the tool can display photographs 
or locations (zones) managed by the system through inte- 
grated hypertext links embedded within crew procedure 5 
documents used to accomplish tasks on-board a space flight 
vehicle. 

The overall system may also include a real-time spaceflight 
trajectory tool, which calculates current and projected trajec- 
tories based on information about the vehicle. In such a sys- 10 
tern, the trajectory tool can monitor information from the 
vehicle, such as attitude maneuvers, altitude changes (e.g. 
away from a celestial body, closer to the celestial body, etc.), 
or actual vehicle position in space to update the vehicle’s 
trajectory. As described herein, by monitoring these trajec- 15 
tory changes, the overall system can dynamically monitor and 
adjust crew schedules to more efficiently use the vehicle’s 
resources and help the crew and ground or support personnel 
deal with and diagnose problems. 

It is to be understood that both the foregoing general 20 
description and the following detailed description are exem- 
plary and explanatory and are intended to provide further 
explanation of the invention as claimed. 

BRIEF DESCRIPTION 25 

The accompanying drawings, which are included to pro- 
vide a further understanding of the invention and are incor- 
porated in and constitute a part of this specification, illustrate 
embodiments of the invention and together with the descrip- 30 
tion serve to explain the principles of at least one embodiment 
of the invention. 

In the drawings: 

FIG. 1 is a block diagram of a computer architecture sup- 
porting an embodiment of the disclosed image and informa- 35 
tion management system. 

FIG. 2 is a block diagram illustrating a conceptual software 
architecture for implementing an embodiment of the dis- 
closed image and information management system. 

FIG. 3 is an alternative block diagram of the architecture of 40 
FIG. 1. 

FIG. 4 is a screen capture of a high-data-level user interface 
as implemented in an embodiment of the disclosed image and 
information management system. 

FIG. 5 i s a screen capture of an intennediate-data-level user 45 
interface as implemented in an embodiment of the disclosed 
image and information management system. 

FIG. 6 is a screen capture of an alternative high-data-level 
user interface as implemented in an embodiment of the dis- 
closed image and information management system. 50 

FIG. 7 is a partial screen capture of an alternative inteime- 
diate-data-level user interface as implemented in an embodi- 
ment of the disclosed image and information management 
system. 

FIG. 8 is a screen capture of a lower-data-level user inter- 55 
face as implemented in an embodiment of the disclosed image 
and information management system. 

FIG. 9 is a screen capture of a low-data-level user interface 
as implemented in an embodiment of the disclosed image and 
information management system. 60 

FIG. 10 is a screen capture of a detailed photograph and an 
image control toolbar for controlling display of the photo- 
graph, as implemented in an embodiment of the disclosed 
image and information management system. 

FIG. 11 is a screen capture of an image control toolbar as 65 
implemented in an embodiment of the disclosed image and 
information management system. 
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FIG. 12 is a screen capture of an image selection user 
interface as implemented in an embodiment of the disclosed 
image and information management system. 

FIG. 13 is a screen capture of an image attribute data entry 
interface as implemented in an embodiment of the disclosed 
image and information management system. 

FIG. 14 is a screen capture of an orb iter/vehicle documen- 
tation log interface as implemented in an embodiment of the 
disclosed image and information management system. 

FIG. 15 is a block diagram of an alternative computer 
architecture for implementing an embodiment of the dis- 
closed image and information management system. 

FIG. 16 is a block diagram illustrating a process through 
which the image and information management system can be 
integrated into business processes. 

DETAILED DESCRIPTION 

Reference will now be made in detail to the embodiments 
of the disclosed image and information management system, 
examples of which are illustrated in the accompanying draw- 
ings. 

FIG. 1 is a block diagram of a computer architecture sup- 
porting an embodiment of the disclosed image and informa- 
tion management system. The illustrated architecture is an 
“n-tier” architecture that is implemented in three tiers, client/ 
presentation tier 100 , application/business logic tier 110 , and 
data/content tier 120. Client/presentation tier 100 represents 
the client computers through which users can interact with 
and retrieve information from application/business logic tier 
110 and data/content tier 120. Application/business logic tier 
110 represents computer hardware and software responsible 
for managing information storage and retrieval, user interface 
generation, and the like. Data/Content tier 120 represents 
physical data storage. 

In FIG. 1, Camera 102 is used to collect photographs of 
various components. These photographs are transferred to 
Computer 104, where the photographs and other images are 
organized and cataloged for storage by eContent Server 112 
as files in a file system 122 and/or a database 124. Once stored 
therein, the pictures are made available to Computers 106 and 
108 via web server 116, which presents data collected from 
eContent Server 112 by Application Server 114 based on user 
requests. 

FIG. 2 is a block diagram illustrating a software architec- 
ture for implementing an embodiment of the disclosed image 
and information management system. In FIG. 2, Application 
Design layer 200 is responsible for controlling access to 
lower layers and for providing an interface through which 
users can retrieve data from database and file system 230. 
Application Design layer 200 is built around model 206, 
which has knowledge of the data structures used by the lower 
layers and is responsible for translating user information 
requests into requests serviceable by the lower layers. View 
204 is responsible for translating information from Model 
206 into an appropriate user interface. Controller 202 man- 
ages the evaluation of information from the user interface and 
translation of such information into a format usable by Model 
206. 

Service Framework 210 provides an interface to the under- 
lying database and file system 230. Service Framework 210 
receives requests from Model 206 and routes them to an 
appropriate data-access interface, such as Documentum® 
Foundation Classes (“DFC”) 212. DFC 212 allows Service 
Framework 212 to access information stored in a traditional 
content management or document management system. An 
advantage to the use of a Service Framework-like layer is the 
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ability to insert new data access “drivers” into the system 
without having to rearchitect the entire system. 

Application Context 220 represents traditional data storage 
and server software and/or hardware that together enable 
solutions for developing, deploying, and managing multi-tier 
server-centric applications. 

FIG. 3 is an alternative block diagram of the architecture of 
FIG. 1. In FIG. 3, Presentation Tier 300 includes request/ 
response interface 310. Request/response interface 310 is 
preferably created by an Hypertext Markup Language 
(“HTML”)/ Java Server Pages server, and can be comprised of 
three main components. The first component, controller 314, 
is responsible for receiving user input from views 318 and 
transferring the information to Data Access 320 (described 
below). Controller 314 also receives information from Secu- 
rity and Session APIs 324 and transfers the information to the 
user via views 318 based on user selections in Filters 312. 

One aspect of controller 314 can communicate with Secu- 
rity and Session APIs 324 via tag libraries 316. Tag libraries 
316 provide controller 314 with information about the under- 
lying data structure used by Security and Session API’s 324. 
Security and Session API’s 324 receives information requests 
from and transfers information to controller 314. Security and 
Session API’s 324 can distinguish between requests for infor- 
mation contained in a relational database (illustrated as 
RDBMS 332) and files stored as part of the file system (illus- 
trated as File System 334) and route the request through an 
appropriate handler, such as Content APIs 326 and Query 
Language 322. 

FIG. 4 is a screen capture of a high-data-level user interface 
as implemented in an embodiment of the disclosed image and 
information management system. In this embodiment, the 
user is presented with an overall image of the object in ques- 
tion 400, in this case, a Space Shuttle Vehicle (“SSV”), exter- 
nal fuel tank (“ET”), and solid rocket boosters (“SRB”). This 
image represents the highest level of the drilldown hierarchy. 
By clicking on one of the SSV, the ET, or an SRB, the user is 
taken to a screen similar to either SSV 420 or ET/SRB 600 of 
FIG. 6, respectively. The images contained in these screens 
represent the next lower level of the drilldown hierarchy. 

Referring again to image 400, if the user is interested in 
accessing information about a portion of the SSV, the user 
selects the SSV, and is presented with a view similar to SSV 
420. SSV 420 can provide a high-level, exploded view of a 
typical SSV, or the image can be used to represent a specific 
SSV configuration based on a particular mission (STS 411) 
chosen by the user from “dashboard” 410. 

FIG. 5 is a screen capture of an intermediate-data-level user 
interface embodiment. By clicking on wing 422 of FIG. 4, the 
display can change to provide an exploded view of the wing, 
as illustrated in block 512. The user can click on a portion of 
the illustrated wing, such as Left Wing Torque Box Leading 
Edge Zone 741, to display a more detailed image, such as that 
illustrated in FIG. 7. FIG. 7 is a partial screen capture of an 
alternative intermediate-data-level user interface embodi- 
ment. Information about a given wing component, such as 
component 710, can be obtained by hovering a pointer over a 
given component, right-clicking on the component, or other- 
wise interacting with the component. Such information may 
be presented in a component-specific information display 
such as component information 720. By double-clicking on 
the component or otherwise interacting with the component, 
the user can cause the display to change to one similar to that 
illustrated in FIG. 8. 

FIG. 8 is a screen capture of a lower-data-level user inter- 
face embodiment. FIG. 8 presents a series of thumbnail pho- 
tographs 810 of component 710 of FIG. 7, the photographs 
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having been taken during assembly, maintenance, or the like 
of the component. By clicking on or otherwise interacting 
with a photograph, the user can cause the display to change to 
one similar to that of FIG. 9. 

5 FIG. 9 is a screen capture of a low-data-level user interface 

embodiment. In FIG. 9, another thumbnail photograph 910 of 
component 710 is presented, along with information 920 
about the thumbnail and the component or components in the 
photograph. Such information may include, but is not limited 
10 to, the date on which the photograph was taken, the vehicle 
into which the component or components are installed, the 
photographer’s name, a description of the contents of the 
photograph, and the like. By clicking on thumbnail photo- 
graph 910, the display can change to one similar to that 
15 illustrated in FIG. 10. 

FIG. 10 is a screen capture of a detailed photograph and an 
image control toolbar embodiment, wherein the image con- 
trol toolbar permits the user to control the display of the 
photograph. By default, the display illustrated in FIG. 10 
20 preferably scales the photograph to the largest size possible 
consistent with the screen or other device on which the pho- 
tograph is being displayed to the user. Panning the photo- 
graph within the display window can be accomplished by 
operating scroll bars associated with the image, by clicking 
25 on the image to bring up a specialized move cursor that 
“sticks” to the image and allows the photograph to be moved 
in any direction within the confines of the display window 
(not illustrated), or other such means. 

30 FIG. 11 is a screen capture of an image control toolbar 
embodiment. As FIG. 11 illustrates, the toolbar may also 
include a unique photograph or image identifier 1105. The 
toolbar may also allow the user to zoom in using control 1120 
or zoom out using control 1110, view the photograph at its 
35 actual resolution using control 1130, and rotate the photo- 
graph using control 1140. The toolbar can also allow the user 
to easily resize the photograph to fit the screen using control 
1135. The toolbar may also allow the user to rotate the pho- 
tograph by fixed amounts 1140. The user an start a new search 
40 based on data attributes of photographs using control 1145. 
The user can report a misfiled or incorrectly classified pho- 
tographusing control 1170, and quickly return to the thumb- 
nail interface illustrated in FIG. 8 by interacting with button 
1150. Interacting with a “close” button or the like (not illus- 
45 trated) can cause the display to return to a default interface 
view, such as that illustrated in FIG. 4. 

The user interfaces, or screens, illustrated in FIGS. 4-11 
allow the user to quickly navigate to all information about a 
component in question via a hierarchical interface. The inter- 
50 face uses the user’s selections as a basis for searching a 
database for information related to the presented image or 
component. To facilitate entry of images, photographs, and 
related information into the system, an embodiment of the 
invention preferably includes well- structured user interfaces 
55 that simplify the organization and data entry tasks. 

FIG. 12 is a screen capture of an image selection user 
interface embodiment. In this interface, the user can select the 
drive or other device 1205 on which one or more images are 
stored and, from that device, the path to the folder or directory 
60 1210 containing the images. List box 1215 enumerates all 
images in the selected folder or directory, and the user can 
select one or more entries for importation into the system. 
Select all button 1218 allows the user to quickly select all 
images listed in list box 1215, and de-select button 1219 
65 clears list box 1215 of all selected images. By clicking cancel 
button 1230, the user can cancel the image importation pro- 
cess. When the user has selected the images to be imported, 
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the user can click download button 1220. The images are then 
imported into the system for further processing. 

FIG. 13 is a screen capture of an image attribute data entry 
interface embodiment. Once the images are imported into the 
system, the user is presented with thumbnails 1305 of each of 
the imported images, and with various tools to assist with the 
importation process. The user can add additional images by 
clicking button 1355, delete a selected image by clicking 
button 1360, undelete an image by clicking button 1365, and 
undelete all deleted images by clicking button 1370. 

The user can also enhance and annotate an image by click- 
ing button 1335. Such enhancements may include, but are not 
limited to, adjusting the lighting levels in an image, adjusting 
the contrast or brightness, correcting the color, and the like. 
Users may be prevented from directly editing image content. 
Further, the system may maintain copies of all images, both 
pre-edit and post-edit, along with a record of the user making 
the edits. If a user is not satisfied with the changes made, the 
user can click button 1345 to revert to the previously stored 
image. The annotations can include comments and notes 
about the image. 

The system allows the user to view a larger version of a 
selected image by clicking button 1340. This can cause the 
display to change to one similar to that of FIG. 10, with a 
toolbar similar to that of FIG. 11. 

The user can also indicate that the image is part of an 
external fuel tank (“ET”) assembly or solid rocket booster 
(“SRB”) by clicking button 1310; part of an orbiter, SSV, or 
other vehicle by clicking button 1315; as a line replaceable 
unit (“LRU”) or flight element set (“FES”) by clicking button 
1325; and as part of the NASA Shuttle Logistics Depot 
(“NSLD”) by clicking button 1320. Although the preceding 
description focuses on clicked buttons, it should be apparent 

1 il j-l ! j. C _1 
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and 1524, or to other information sources and services, such 
as those provided by internal web services 1521. In this role, 
application server 1514 can receive information about user 
interaction with an interface provided by web server 1512, 
5 such as the user clicking on a component in an image, and can 
translate such information into data requests in an appropriate 
query language for interacting with and retrieving informa- 
tion from the databases and internal web services 1521. 

EIS Tier 1520 represents back-end systems to which 
10 middle tier 1510 provides an interface. EIS Tier 1520 
includes systems such as a database associated with an exist- 
ing content management system 1524, as well as trajectory 
tracking, resource planning and other internal systems and 
applications 1523, 1525, and 1526. Internal web services 
1521 provides a standardized interface through which appli- 
1 5 cation server 1514 and other hardware and software can inter- 
face with these internal systems. Adapters 1522 can provide a 
further layer of abstraction, allowing the internal systems to 
be easily upgraded or replaced without necessitating rearchi- 
tecting the entire system. 

20 FIG. 16 is a block diagram illustrating a process through 
which the image and information management system can be 
integrated into business processes. Although FIG. 16 focuses 
on integration of the image and information management 
system into a spaceflight related business process, it should be 
25 apparent to one skilled in the art that the process can be 
readily adapted to allow the image and information manage- 
ment system to be integrated into alternative business pro- 
cesses without departing from the spirit or the scope of the 
disclosed image and information management system. In 
30 FIG. 16, a set 1600 of standard photographic or other images 
for an object or component is defined. The definition may be 
set forth in conjunction with a Work Authorization Document 
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When the images are stored in image repository 1635, the 
photographs are available 1640 for study and use by autho- 
rized persons. Such use can include, but is not limited to, 
permitting support personnel 1650 to access the images in 
conjunction with on-orbit troubleshooting and operational 5 
needs 1645. 

While the invention has been described in detail and with 
reference to specific embodiments thereof, it will be apparent 
to those skilled in the art that various changes and modifica- 
tions can be made therein without departing from the spirit 
and scope thereof. Thus, it is intended that the present inven- 
tion cover the modifications and variations of this invention 
provided they come within the scope of the appended claims 
and their equivalents. 

The invention claimed is: 

1. An information management system comprising: 

at least one image of an object, wherein the object is com- 
prised of a plurality of components; 

a plurality of images of at least a subset of the plurality of 
components; 

at least one first data table, wherein the first data table stores 20 
at least one relationship between the plurality of com- 
ponents and the object; 

at least one second data table, wherein the second data table 
stores associations between the at least one image of the 
object and the object; 25 

at least one third data table, wherein the third data table 
stores associations between the plurality of images of 
the plurality of components and the plurality of compo- 
nents; 

at least one fourth data table, wherein the fourth data table 30 
stores an identified at least one actionable region asso- 
ciated with the at least one image of the object, and 
wherein the fourth data table associates at least one 
action with each of the stored at least one actionable 
regions; and, 

a user interface, wherein the user interface displays at least 
one of the plurality of component images or the at least 
one object image, wherein the user interface monitors 
user interaction to determine when a user activates a user 
actionable region. 

2 . The system of claim 1 , wherein the second data table and 40 
the third data table are part of the same data table. 

3. The system of claim 1, further comprising at least one 
database. 

4. The system of claim 3, wherein the first data table, the 
second data table, the third data table, and the fourth data 45 
table, are stored in the at least one database. 

5. The system of claim 3, wherein the plurality of images of 
at least a subset of the plurality of components and the at least 
one image of the object are stored within the at least one 
database. 

6. The system of claim 1, further comprising at least one 
fifth data table, the fifth data table storing an identified at least 
one actionable region and permitting the at least one action- 
able region to be associated with at least one of the images. 

7. The system of claim 6, further comprising a sixth data 
table, wherein the sixth data table stores at least one associa- 
tion between information about at least one of the plurality of 
components and the plurality of components. 

8. The system of claim 1, wherein the user interface causes 
the information about at least one of the plurality of compo- 
nents to be displayed when an actionable region associated 60 
with a component is activated. 

9. The system of claim 1, wherein the user interface further 
comprises a dashboard. 
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10. The system of claim 1, wherein the user interface 
changes based on the information to be presented therein. 

11. The system of claim 1, the user interface comprising a 
toolbar when a component image is displayed, wherein the 
toolbar facilitates zooming in and out and reporting misfiled 
images. 

12. The system of claim 1, wherein the first data table 
further stores the relationship between each component and 
any sub -components thereof. 

13. The system of claim 12, wherein the first data table 
stores the relationships in an hierarchical data structure. 

14. A method of presenting information to a user, compris- 
ing: 

receiving at least one image of an object; 
receiving a plurality of images of a plurality of compo- 
nents, wherein the object comprises the plurality of com- 
ponents; 

storing first information about at least a subset of the plu- 
rality of components; 

storing second information about the object; 
associating the stored first information with an appropriate 
one of the plurality of components; 
associating at least one first actionable region with at least 
a subset of the at least one images of the object; 
associating an action with the at least one first actionable 
region; 

associating at least one second actionable region with at 
least a subset of the plurality of images of the plurality of 
components; 

displaying for a user at least one image of the object; 
monitoring actions made by the user to determine when 
one of the at least one first actionable regions has been 
activated and performing the action associated with the 
activated first actionable region; 
displaying for the user at least one of the plurality of com- 
ponent images; and, 

monitoring actions made by the user to determine when 
one of the at least one second actionable regions associ- 
ated with the displayed at least one component images 
has been activated. 

15. The method of claim 14, further comprising associating 
an action with the at least one second actionable region and 
performing the action associated with the activated second 
actionable region when one of the at least one second action- 
able regions has been activated. 

16. The method of claim 15, wherein the at least one action 
associated with a second actionable region comprises dis- 
playing information about the component in the image con- 
taining the second actionable region. 

17. The method of claim 14, wherein the displaying of at 
least one of the plurality of component images and the moni- 
toring of user actions to determine when one of the at least one 

50 actionable regions associated with the displayed at least one 
component images has been activated are iteratively repeated, 
thereby permitting the user to drill down to a desired compo- 
nent. 

18. The method of claim 14, further comprising displaying 
55 a toolbar when the at least one of the plurality of images of the 

plurality of components is displayed for the user. 

19. The method of claim 16, wherein the toolbar permits 
the user to at least zoom in, zoom out, and rotate the displayed 
image. 

20. The method of claim 14, further comprising displaying 
a dashboard when any of the at least one images of the object 
are displayed. 





